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THE SSM/PMAD AUTOMATED TEST BED PROJECT 


In conjunction with MSFC's Work Package One responsibilities 
and MSFC’s previous OAET work in electrical power system 
autonomy, the SSM/PMAD autonomous subsystem project was 
initiated in 1984. The project's goal has been to design and 
develop an autonomous, user-supportive PMAD test bed 
simulating the SSF Hab/Lab module (s). Funded primarily by the 
SSF Advanced Development Program from FY85-88 and with 
additional joint funding from OAET (Code RC) during FY89-91, 
an eighteen kilowatt SSM/PMAD test bed model with a high 
degree of automated operation has been developed. To date 
over $3.2 million has been invested in hardware and software 
development. This advanced automation test bed contains three 
expert/knowledge based systems that interact with one another 
and with other more conventional software residing in up to 
eight distributed 386-based microcomputers to perform the 
necessary tasks of real-time and near real-time load 
scheduling, dynamic load prioritizing, and fault detection, 
isolation, and recovery (FDIR) . 

The approach has been to establish the technology through key 
"operational" demonstrations, prepare for early ground-based 
implementation in the various SSF control centers, and then 
to migrate the technology "on-board" as confidence builds and 
as schedules permit. A parallel effort was begun to establish 
communication links between the SSM/PMAD test bed and the 
primary PMAD automated test bed at LeRC in order to 
investigate major automated subsystem interactions. A first 
generation "operational" prototype has been successfully 
demonstrated along with a Phase I MSFC/LeRC test beds 
communications link. 
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SUMMARY 



OUTLINE 


This presentation will begin with a look at the PMAD 
"problem*' and include a short background and history of the 
SSM/PMAD project. Next, the objectives of the project will be 
presented followed by a discussion of the technical approach 
and description of the project. The next topics will consist 
of how the project integrates with the baseline program and 
how the technology may evolve into the "on-board" station. A 
final summary will then be presented. 
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TYPICAL HOUSE PMAD 
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TYPICAL HOUSE PMAD 


The typical house PMAD system consists of a power meter, a 
circuit breaker box, and the various loads and outlets (a few 
of which have ground fault protection) . The power feed 
(240/120 Vac, 60 Hz) comes from a transformer mounted on a 
pole or underground through a Watt-hour (Energy) meter into a 
control box consisting of a group of circuit breakers (and in 
some cases fuses) . This power is then distributed through the 
breakers to the various loads. Typical loads are: electric 
oven, clothes dryer, heating/air conditioning, water heater, 
lighting, and the outlets. 

This system contains easily accessible circuit breakers 
(fuses) which are electrically simple electomechanical 
devices. Their trip levels (amount of current required to 
“open" the device) are set high which means a major fault or 
an extraordinary number of loads in a particular outlet is 
required to cause the breaker to trip. This is done to 
prevent a user from having to spend his entire life resetting 
breakers. The system allows complete load flexibility. The 
only requirements for any load is to be of the correct 
voltage/ frequency and to draw less current than the breaker 
setting. Finally, the use of energy in this system is cost 
managed. If you can afford it, you can use as much energy as 
you like. 
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TYPICAL SPACECRAFT POWER SYSTEM 

Power Busses 



GO 
"O 
O & 

h- q 



E p 

O .i= 

o o 


892 


Complex Loads with Unique Power Profiles 
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TYPICAL SPACECRAFT POWER SYSTEM 


The typical spacecraft power subsystem to date has been less 
than 5 kW total power and has consisted of two or more power 
channels (solar array, battery, and bussing) feeding power 
distributors to the specialized loads. The systems have been 
primarily low voltage dc (28 Vdc nominal) distribution using 
planar silicon solar arrays and Nickel based batteries. 

In comparison to the terrestrial power system, the spacecraft 
power system has electrically complex circuit breakers with 
very little access capability (if at all) . These breakers are 
also sized for practically each specialized load. The loads 
themselves typically have complex power profiles (power vs. 
time) and require an extensive scheduling team to combine the 
power profiles to maximize energy usage. This leads to the 
final point. A spacecraft has to be load managed in order to 
maximize its most precious resource - energy. For example, if 
a load begins to use more energy than its initial allotment, 
then either this load is shed or energy is shifted from a 
more efficient load or another load is shed. 
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AUTOMATION PROJECTS AT MSFC 
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OSF and OAET Funded 



AUTOMATION PROJECTS AT MSFC 


As background, there have been three primary automation 
projects at MSFC. The first, AMPS, was started in the late 
70 's to investigate large spacecraft power systems and how to 
automate them. The AMPS project, funded by OAET (Code RP) and 
contracted to TRW, consisted of three phases. Phase I 
identified a reference 250 kW-class power system based on 
projected 1980's technology. The basic result was a 
distributed multiple power channel system, using concentrator 
solar arrays. Nickel -Hydrogen batteries, and high voltage dc 
(200 Vdc nominal) distribution. Phase II focused on how to 
automate the system. The basic results were using distributed 
microcomputers and pushing the computing power as far down 
the architecture as possible. Phase III involved constructing 
a three power channel (25 kW) subsection of the reference 
power system to design and demonstrate the automation 
theories. The project was stopped shortly before full 
completion, but the basic automation theories were able to be 
demonstrated . 

The second project area was the Hubble Space Telescope power 
system test bed. This project area introduced MSFC into the 
area of expert/knowledge based systems. Two separate systems 
were developed to automate and perform fault diagnosis on the 
HST power test bed which was (and is) operating 24 hours a 
day. When a test bed problem occurs, the system is safed and 
the test engineer is automatically called. During the travel 
time of the engineer to the test, the expert system has 
analyzed the situation and produced a diagnosis and 
explanation before the engineer arrives. These systems also 
are able to do multiple orbit trends analyzes. These systems 
were named the Nickel Cadmium Battery Expert System (NICBES) 
and the Nickel Hydrogen Battery Expert System (NIHES). The 
two systems were a result of the HST battery change in 1989. 

The final active project area is the SSM/PMAD automation test 
bed project which is the topic of this presentation. The 
project was started in 1985 with funding from the Phase B 
advanced development program for Space Station. Martin- 
Marietta of Denver was awarded both the hardware and 
automation contracts. Presently, the project is funded 
through the Advanced Development Office of Space Station 
(Code MT) with OAET (Code RC) funding two subtasks which are 
more research in nature. 
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OBJECTIVES OF THE SSM/PMAD 
AUTOMATION PROJECT 
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OBJECTIVES OF THE SSM/PMAD PROJECT 

The objectives for the project are to provide risk reduction 
for the Space Station Hab/Lab Modules power subsystem and to 
identify any design impacts both to the baseline and the 
evolution station. 

The first objective is being met by having designed a high 
fidelity hardware test bed of the power subsystem and then to 
demonstrate autonomous control through the use of advanced 
and conventional software. The basic system has been designed 
and operational -type testing is being performed to evaluate 
and update the software/hardware. 

All information from the design and test is made available to 
all SSF work packages, but especially to WP01 and WP04 . This 
information is used to help guide design decisions for the 
baseline station. In addition, this information can be used 
to help in module power subsystem operations and to aid in 
future hardware/ software upgrades to the evolving station. 


897 




898 



BENEFITS OF AUTOMATED PMAD 

Automating the SSM/PMAD subsystem will produce many benefits. 

Five of these benefits are listed below: 

(1) Safety is enhanced through the use of fast, intelligent 
hardware which can safe faults rapidly. Also, a critical 
load which loses power during a fault can be re-powered 
in a few seconds (less than 3) using a dual power feed 
and a small, but efficient computer. 

(2) Productivity is increased by allowing the power system 
operator (ground or flight) to focus on more critical 
tasks than the operation of the power subsystem. Through 
the use of dynamic re-scheduling, even in off-nominal 
situations, the source energy to load energy ratio can be 
maximized. 

(3) Skylab required twenty ground support personnel and a 
flight crew of three to operate an 8 kW power system. 
Using automation techniques, as SSF evolves, the number 
of personnel required to operate the power system can 
remain constant. Further, as the user interface matures, 
the technical expertise required by an operator could be 
reduced . 

(4) Reliability is increased by the system consistency 
offered by the automated software. Also, system hardware 
stress is reduced through intelligent load scheduling and 
load energy balancing. 

(5) System faults are safed, isolated, and diagnosed iemm. in 
a few milliseconds to seconds which allows for quicker 
repair and reduced downtime. 
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TECHNICAL APPROACH 
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TECHNICAL APPROACH 


The technical approach to the project was to build a test bed 
model of the SSM/PMAD subsystem and then to add the 
automation software and any additional hardware needed for 
full autonomous operation. The automation software would be a 
combination of standard software and the latest advanced 
software techniques. The present software architecture 
consists of three expert/knowledge based systems and numerous 
specialized conventional programs. 

One of the first steps taken was to analyze the power system 
operation process and then to break these processes into 
their various functions. The next step was to arrange these 
functions according to their time criticalities and then to 
distribute the functions in such a way as to maximize their 
speed. Thus, the critical time functions are performed 
nearest the loads using conventional software with the less 
time critical functions being performed further from the 
loads, but using more powerful hardware/software tools. 

The last key to the project was to use a powerful user- 
supportive graphics interface to allow for the fourth expert 
in the system, namely, the human system operator. The 
interface has become an integral part of the operation of the 
system as well as providing valuable information as to how 
the system is determining its control decisions. 
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HARDWARE/ SOFTWARE SCHEMATIC DIAGRAMS OF THE SYSTEM 


This test bed hardware has two power distribution control 
units (PDCUs) and three load centers. The basic system 
design allows for two additional load centers. Further, the 
test bed includes remote bus isolators (RBIs) , remote 
controlled circuit breakers (RCCBs) , and remote power 
controllers (RPCs) . Lastly, a lowest level processor (LLP) is 
included in each PDCU and load center. In the software area 
of the test bed, autonomy is pushed down to the lowest 
levels, specifically, to the LLPs and through the switch 
interface processors to the "smart” switchgear. Three 
Artificial Intelligence (AI) systems - the Fault Recovery And 
Management Expert System (FRAMES) , the Load Priority List 
Management System (LPLMS) , and the Master of Automated Expert 
Scheduling Through Resource Orchestration (MAESTRO) -reside 
above and communicate with the other processors through the 
Communications and User Interface (CUI) software. 

The system software is distributed through several 
different types of processors and at different hierarchical 
levels. The LLPs are located at the level nearest the power 
hardware. The CUI software is notified of any anomalies by 
the LLP. FRAMES, MAESTRO, and LPLMS share the highest level 
of the hierarchy. Each step up this hierarchy reveals a 
decrease in speed (microseconds at the switchgear level, 
milliseconds to seconds at the LLP level, seconds to minutes 
at the AI level and an increase in sophistication. 

The LLPs consist of Intel 80386 based computers and an 
Ethernet communication board. A LLP is located in each load 
center, subsystem distributor, and PDCU. Each LLP is 
responsible for controlling the switches associated with it 
and for keeping track of all the sensor readings and switch 
positions in its center. The LLP also executes scheduled 
changes in switch positions, sheds any loads which exceed 
their scheduled maximum, and switches redundant loads to 
their secondary bus if the load's primary source is 
interrupted. The LLP passes any or all of this information to 
the CUI software. 

The CUI software is resident in a Solbourne 5/501 UNIX 
based workstation. The CUI software routes information to the 
various LLPs, controls LLP initialization, and serves as the 
man/machine interface for the entire system. Messages are 
passed from the three AI systems to the LLPs through the CUI 
via Ethernet communication links. 
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SSM/PMAD MANAGEMENT AND SOFTWARE LOGIC FLOW DIAGRAM 
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SCHEMATICS DESCRIPTION (CONTINUED) 


The FRAMES resides on the Solbourne 5/501 workstation 
and is implemented in the Common Lisp Object System (CLQS) . 
This expert system watches over the entire EPS looking for 
anomalies and failures. FRAMES is responsible for detecting 
faults, advising the operator of appropriate corrective 
actions, and, in cases involving critical loads, autonomously 
implementing corrective actions through power system 
reconfigurations. FRAMES recognizes and adjusts to hard 
faults which the smart switchgear handles immediately, as 
well as handling soft faults, cascaded faults, and 
independent multiple faults. 

The LPLMS resides on the Solbourne 5/501 workstation and 
is implemented in LISP. The LPLMS keeps track of the dynamic 
priorities of all payloads while developing and downloading 
current load shedding lists for the LLPs every fifteen 
minutes in preparation for contingencies which necessitate 
load shedding. This way, load shedding is implemented 
quickly in each load center or subsystem distributor. The 
LPLMS maintains a real time dynamic representation of all the 
module loads and relevant facts so that applicable rules can 
fire to reorder portions of the load shedding list as 
situations change. The loads in a laboratory module may have 
dynamic properties. A critical noninterruptible materials 
processing experiment involving crystal growth will 
undoubtedly have a different priority as it nears completion. 
Other factors may change priorities such as equipment 
malfunctions. An expert system such as the LPLMS is crucial 
in determining which loads must be shed in the event of 
perturbations to the available power. The LPLMS insures that 
critical loads not be shed unnecessarily. 

MAESTRO resides on a Symbolics 3620D and is implemented 
in LISP. Special interfaces have been developed for MAESTRO 
which allow a great deal of flexibility in interactions with 
the scheduler. MAESTRO is a resource scheduler developed by 
Martin Marietta and can schedule and reschedule a number of 
payloads with various scheduling constraints. This AI system 
generates the baseline schedules for the EPS and accepts 
information from the other processors on when and how to 
reschedule module payloads. MAESTRO uses pieces of several AI 
technologies including object-oriented programming, 
heuristically guided search, activity library, expert 
functions, etc. MAESTRO schedules loads with regard to 
numerous resource constraints such as available crew members, 
supplies for payloads, interdependence of payloads, power 
profiles, and thermal status. 
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SCHEMATICS DESCRIPTION (CONTINUED) PLUS PHOTOGRAPH 

In order to efficiently operate these three expert 
systems together, a simultaneous multi-agent knowledge 
manager function called the Knowledge Management and Design 
(KNOMAD) system was designed and built . KNOMAD utilizes a 
distributed database management function to provide a 
modified blackboard management capability. The KNOMAD 
architecture is layered. The central layer is the database 
which provides a place for storing working memory data, for 
transferring and sharing data, and for storing long term 
data. The database is modular and may be implemented as a 
distributed database. As a distributed database, multiple 
cooperating knowledge agents, each in different physical 
locations, could be supported. The next layer consists of an 
interface to the database that provides a frame system for 
abstracting both data and procedure as well as a mechanism 
for storing simple facts. The top layer is the place where 
various tools are defined and implemented. All of the tools 
make use of the same data representation and thus easily 
share data across domains and functions. FRAMES was 
implemented in KNOMAD in June of 1990 with LPLMS and a 
MAESTRO interface being implemented in April 1991. 

PHOTOGRAPH 

In the front of the photo, the Solbourne workstation is on 
the right and the Symbolics AI workstation is on the left. 
Looking at the racks, The PDCU racks are on the right with 
the three load center racks to the left of the PDCUs. The far 
left rack consists a few representative loads. The majority 
resistive loads are located in an annex building to this 
room. Each rack, from top to bottom, consists of an LLP, a 
group of 1 kw or 3 kw RPCs, a group of RPC controller cards 
(behind the silver plate) , housekeeping power supplies, and 
cooling fans. Load Center 2's (Material Science Rack) LLP is 
located on a table to the right of the Solbourne (easier 
access) . 
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USER INTERFACE PHOTOGRAPH (POWER SYSTEM SCREEN) 


This photo of the user interface features the power system 
screen (PSS) which is the primary screen for normal 
operation. Located in the center right window, the PSS 
displays power flow through the use of white filled in 
"pipes" and RPC open/close through a toggle switch icon. The 
l kW RPC rectangles and the 3 kW RPC ovals are colored green 
for nominal operation, red for faulted conditions, and brown 
for out-of-service. They also display their designator (small 
print) and the amount of current flow through the RPC (bold 
number) . The diamonds represent the RBI and the small circles 
represent additional voltage and current sensors. The 
selection rectangle to the left of the PSS is for obtaining 
more detailed information for each or all RPCs. When 
requested, this information is displayed underneath the PSS 
in the "scratch-pad" window. The various modes (Ready, 
Created, Manual, Autonomous) are displayed above and to the 
right of the PSS. Various Utility, Function, and Help 
requests are made through pull down windows just above the 
PSS. Located to the left center of the PSS, the KNOMAD screen 
dynamically displays Ethernet connections, the various 
application programs, and their status. The message screen (s) 
at the bottom left gives textual data for the messages being 
passed through the system. Through the Utilities menu, a 
Focused Message window can be brought up which displays 
filtered messages as chosen by the user. The final active 
screen is the Screen Selection window in the upper left 
corner of the interface. When selected, one of four sub- 
screens replaces the PSS for further information. 
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USER-INTERFACE (FELES SCREEN) 

This is a photo of the FELES screen selection. This screen 
shows a timeline for each scheduled load and a marker showing 
the present time on the schedule. Again, additional 
information can be requested through the rectangle box to the 
left of the FELES screen. 
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USER- INTERFACE (POWER UTILIZATION SCREEN) 


This photo displays the Power Utilization screen which 
displays a power versus time load profile for each PDCU, load 
center, and the individual loads. The white marker displays 
the present time and the white line displays actual power 
usage versus time. 
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BASELINE INTEGRATION 
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Establish Key Relationships with Baseline Personnel 


BASELINE INTEGRATION 


As mentioned earlier, one goal of this project is to maintain 
as close of ties as possible to the baseline design of the 
SSF. Listed below are a few of the ways in which this goal is 
being accomplished. 

(1) As much as possible, we are attempting to make our 
testbed mimic the baseline PMAD testbed. In most cases, 
this will require us to disable many of the advanced 
features of the original testbed, especially in the RPCs, 
the sensors, and the lower level processors. 

(2) We are presenting operational demonstrations before the 
critical design reviews in order to provide more and 
better data to guide decision making. The first 
operational demonstrations are being held this summer 
with a second more advanced demonstration to be held next 
summer. The CDRs for the WP01 PMAD are scheduled for 
early 1993. 

(3) We are planning for an early ground-based implementation 
in order to support the POCC at MSFC, the ESC at MSFC, 
and the SSOC at JSC. Implementation could be accomplished 
by porting real-time or near real-time flight data into 
the SSM/PMAD computers and then perform system fault 
diagnosis with both the ground hardware data and the 
flight data. 

(4) We have completed a Phase I link with a LeRC automated 
test bed. A simple fault handling scenario was then 
successfully demonstrated. This will form the basis for a 
full LeRC SSF automated test bed/MSFC SSF automated test 
bed link to be completed late in 1992. This will allow 
for ground system testing between the two major power SSF 
subsystems . 

(5) Relationships are being established with all key baseline 
personnel. These include, but are not limited to: MSFC 
and Boeing power system design engineers, MSFC and Boeing 
SSF project offices, LeRC and Rocketdyne power system 
design engineers, MSFC and Boeing system integrators and 
operations personnel, JSC mission control system 
personnel , and various Level 1 and 2 personnel at NASA 
Headquarters . 
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GROWTH AND EVOLUTION OPTIONS 


In order to meet our goal of on-board automation, a series of 
key and orderly steps to flight are being planned. The first 
step is to establish the ability to automate the actual SSF 
PMAD through ground based implementation. The next step would 
be to act as a power system engineer surrogate through the 
use of powerful portable computer workstations being 
designed. The automation software could be downloaded into 
the workstation, flown to SSF, and then attached to the on- 
board data stream. A next step would be to retrofit 
" intelligent” RPCs which are now being designed. A final step 
would be to incorporate the automation equipment more 
permanently by mounting the system in new rack(s) and 
performing a rack(s) retrofit. 
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SUMMARY 
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Evolves to "On-Board" Operation 



SUMMARY 


This paper has described the various activities at NASA/MS FC 
for advancing the state-of-the-art in spacecraft electrical 
power system automation. Based on the AMPS and SSM/PMAD 
projects, a hierarchical approach of distributed processing 
is being developed. In addition, AI and in particular, 
knowledge-based systems, are proving to be invaluable in 
accomplishing tasks not possible with conventional software. 
We are demonstrating PMAD risk reduction through the use of 
autonomous monitoring, control, and FDIR. Basic concepts have 
been established with a first generation operational 
prototype having been successfully demonstrated. The next 
steps Involve integrating the testbed into the ground based 
support centers and then evolving onto the SSF. Thus, 
NASA/MSFC is progressing toward the eventual goal of a 
totally autonomous power system (with human override) . 
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